Semi - supervised Learning of Instance - level Recognition from Video

Over the recent years, data-driven approaches that take advantage of the availability of large manually-annotated datasets have proven effective at visual perception tasks. We witnessed particularly rapid improvements in the image classification task, where machines now surpass human-level performance. However, the progress in certain instancelevel recognition tasks, such as object segmentation and human pose estimation, has not been as rapid. In part, the rate of improvement has been affected by the difficulty of manually annotating very large amounts of data for these tasks. In this work we explore if semi-supervised learning approaches that utilise unlabelled video data can provide a reasonably effective alternative to the manually annotated data for instance-level recognition. Central to our approaches is the idea that good models should consistently predict the same label for different pose and view variations of the same object instance. We employ a hard example mining heuristic to find video frames in which the model makes mistakes and correct them by combining the information from the remaining video frames. By noting that video can be seen just like a source of transformation, we generalise our approach to unlabelled images and apply it to the human pose estimation task. The resulting technique, which we call keypoint data distillation (KDD), is simple and very effective. Using a collection of unlabelled video frames, we show that the Mask R-CNN model combined with KDD achieves state-of-the-art results on the COCO Keypoint Challenge, outperforming all other entries by a significant margin.